research
Networks
Scaling Up
in
of schools facilitate higher student achievement
Chile
On international tests, Chilean students in 2006 outperformed those of all other Latin American countries in reading and were second only to Uruguay in math (see Figure 1). But although Chile’s educational performance appears to outstrip that of its closest competitors, the country’s educational system has become highly controversial among scholars throughout the western hemisphere. By and large, the education systems of most Latin American countries are all but ignored by outside scholars. However, the Chilean system has generated a veritable cottage industry of research scholarship that has yielded a range of conflicting findings. The explanation for this odd fact: since 1981 Chile has had a more comprehensive school choice system than any other country in the world, as well as a system of publicly available information on student test performance. Scholars have thus seen Chile as a place to test theories of school choice. Do students with vouchers learn more in private schools or in those run by municipalities? What is the impact of a voucher system on equality of educational opportunity? The answers to these and related questions have been just about as varied as the number of scholars who have inquired into the matter. On balance, the bulk of the research shows a small educational advantage for students who attend privately operated voucher
BY GREGORY ELACQUA, DANTE CONTRERAS, AND FELIPE SALAZAR
62
E D U C AT I O N N E X T / S U M M E R 2 0 0 8
www.educationnext.org
research
CHILEAN SCHOOLS ELACQUA ET AL.
And the Winner Is…
(Figure 1)
Chile’s education system leads Latin America in reading, and rates second-best in math.
Math Reading schools rather than municipal ones. But hardly any study looks at differences among the voucher schools, and none has examined differences between private schools in netOECD Countries works and those that operate on a stand-alone basis. Yet interest in school networks has escalated since many operators of charter schools in the United States have begun Chile to expand their operations beyond a single school. Some have argued that this is the ideal way for connecting school choice to school improvement. If effective schools Uruguay can expand either by setting up or by “franchising” other schools, school quality can gradually improve. But others say that the formation of networks of schools will lead to a standardization that will undermine the vitality of indiMexico vidual school communities. Chile is an ideal place for exploring these questions. In 2002, only 53 percent of students were still being educated Brazil in municipally run schools, which nonetheless received a good deal of their funding from the vouchers paid for by the national government. Another 9 percent of students Colombia attended fee-based private schools that were independently operated and received no government assistance whatsoever. For the most part, these were schools with wellArgentina established reputations that served the country’s upper class. The remaining students attended what might be called voucher schools, because the schools, while private, 200 250 300 350 400 450 500 550 had been since 1981 heavily dependent on the subsidy that Average score in 2006 the schools received from the national government for each student they enrolled. This sector is the fastest growing segSOURCE: Organisation for Economic Co-operation and Development (OECD) Program for International Student Assessment (PISA), 2007 ment of the Chilean educational system. Like American charter schools (see “Brand-Name Charters,” features, page 28), Chile’s privately run voucher about 20 percent of primary (K–8) private voucher school stuschools may be part of a larger organization or school network, dents attend schools that belong to networks that have more or operate on their own. Most schools are of the stand-alone than three schools (see sidebar, page 65). or “mom-and-pop” variety: 25 percent of all students in Chile Proponents of school networks say that the networks facilattend such schools. But another 13 percent of students attend itate the flow of information (such as research on best practices) schools that are part of a network of two or more schools. among schools and provide political benefits, credibility, and legitThe schools, inside and outside of networks, vary from one imacy in the eyes of the community. They argue that larger another in many ways. Some are operated by teachers who once schooling operations have more access to private investments worked in municipal schools. Others are run by business and loans to expand than smaller operations do. Supporting this entrepreneurs. Fifty-nine percent of the network schools are view, research on public charter schools in the United States indirun by nonprofit entities, either religious or secular. For-profit cates that well-established charter school networks can build credorganizations operate the remainder. Some schools as well as ibility for fund-raising more easily than stand-alone charter some networks are religious. Most networks, and especially schools can. The underlying hypothesis is that, all else bring equal, those in rural areas, consist of just two or three schools. Only the more a schooling organization facilitates transactions between
www.educationnext.org
S U M M E R 2 0 0 8 / E D U C AT I O N N E X T
63
members of a school’s community, the better the school’s performance. The research conducted so far shows that stand-alone charter and brand-name schools, like their district counterparts, vary widely in quality. Critics of school networks fear unintended negative consequences. They argue that large centralized operations create hardto-manage bureaucracies and make it difficult to maintain order and create a sense of community among students, parents, teachers, and administrators. Opponents also claim that large schooling operations grant too much authority to administrators and other professionals far removed from the classroom. Some critics are concerned that consolidation encourages standardization. For instance, they maintain that school networks must establish a brand to be successful, which Both network and stand-alone voucher-subsidized schools have been operating for necessitates relatively uniform operations several decades. and services from site to site. They argue that the administration of schools, transferring responsibility this branded approach to education stifles innovation. for public school management from the Ministry of EducaVery little factual information is available to sort out the tion to municipalities (recognized neighborhoods in Chile credibility of these claims and counterclaims. It is thus of around which municipal services are organized). The govinterest to examine the Chilean experience, where both network ernment also changed the education financing scheme. and stand-alone voucher-subsidized schools have been operMunicipalities began to receive funding from the central ating for several decades. Information on more than onegovernment according to the number of students who chose quarter million students who were 4th graders in 2002 allows to attend the municipal schools. Any enrollment loss had a us to compare Spanish language and mathematics achievement direct effect on their education budgets. Equally important, in network and stand-alone voucher-subsidized schools. Our privately run schools that had not charged tuition began findings suggest that network schools in Chile are more effecreceiving the same per-student voucher as the public schools. tive than stand-alone schools, and that larger networks tend to Tuition-charging (elite) private schools mostly continued to operate without public funding. Despite the parity in funding, significant differences remained between municipally run schools and privately run voucher schools. First, starting in 1994 municipal elementary schools were not allowed to charge parents fees, while all privately run voucher schools could. Second, outperform smaller networks. While we cannot be certain municipal schools were required by law to accept all who whether the higher performance of network schools is because applied. Private voucher schools, in contrast, were allowed good schools were the ones to expand or whether networking, to consider results from admissions tests and parent interby itself, had a positive impact, our results nonetheless add conviews when making admission decisions. Third, municipal siderably to the sparse information currently available on a quesschools had to comply with labor laws that made it virtution of substantial policy interest. ally impossible to fire a low-performing teacher. Privately run voucher schools had greater freedom to terminate School Reform in Chile employment. In addition, municipal school teachers During the 1980s, the Chilean government decentralized received salary increases incrementally, based on years of
Our findings suggest that network schools in Chile are more effective than standalone schools, and that larger networks tend to outperform smaller networks.
64
E D U C AT I O N N E X T / S U M M E R 2 0 0 8
www.educationnext.org
research CHILEAN SCHOOLS ELACQUA ET AL.
experience. There were no rules with regard to incremenBecause we lacked complete data for some schools, our tal salary increases in the private school sector. study includes 252,202 students. Fifty-eight percent of the stuThe policies sparked a movement of students from municdents were attending municipal schools, 24 percent were ipal to previously existing private schools as well as the estabattending stand-alone schools, and 18 percent were attending lishment of new institutions. In 1981, 15 percent of the nearly network schools. In addition to separating out municipal 2.9 million Chilean K–12 students had been attending private and stand-alone private schools, our analysis subdivides the schools that received some public subsidy, and another 7 pernetwork schools into five groups—those that are in networks cent attended elite, unsubsidized private schools. By 1990, 34 of two, three, four, five, and more than five schools—for a total percent of students attended privately run voucher schools; by of seven categories. (We have excluded information from the 2002, enrollment in such schools reached 38 percent of the elite, independent schools that receive no government subsidy.) roughly 3.4 million in total enrollment (see Figure 2). The trend We compare the test scores of students in each of the seven continued after 2002, the year in which the data for this study categories, taking into account differences in the students’ were collected. By 2005, 43 percent of students were enrolled in privately run voucher schools. As indicated above, about one-third of the The Many Faces of Voucher Schooling voucher schools belong to networks, while the remaining two-thirds operate independently. Schools that receive government vouchers in Chile vary from for-profit netBeginning in 2003, after our data were colwork schools to small stand-alone schools. Here are three examples: lected, the Chilean government sought to alter For-profit network Sociedad Educaciónal Tte. Dagoberto Godoy Ltda. several features of the system, although not all operates seven schools: two are located in poor municipalities, four are in of the changes have been fully implemented: lower-middle-class municipalities, and one serves a middle-class municipalRather than providing vouchers at a flat rate, ity in Santiago. Owner Walter Oliva’s parents were teachers who founded voucher amounts are to be tied to family income. most of the network’s schools in the 1970s and early ’80s. A successful Private voucher schools can no longer select entrepreneur, Oliva also has business investments in agriculture; he manstudents in primary school; for secondary school ages the schools and his other businesses from his headquarters in Santiadmission, they can administer tests, but they ago. Standardized test scores for these schools are high compared to the cannot conduct parent interviews. In addition, national average and very high compared to schools with similar students. Congress recently passed legislation that will Nonprofit Catholic network Congregación Salesiana operates thirteen hold schools accountable for student achieveschools: five in Santiago, four in the south of Chile, and three in the central ment and improvement over time. part of the country. The smallest school in the network serves around 600 The political debate continues. In 2006, students, the largest more than 1,700. Although most Congregación Salewidespread student protests of inequalities in the siana schools outperform similar schools in Chile, test scores vary widely education system prompted debate over across the network. whether entrepreneurs should be able to own For-profit stand-alone school Franz Liszt Nº 784 serves 240 students in and run private voucher schools for profit. ProMaipú, a middle-class municipality in Santiago. Owner Marina Goméz Bustaposed legislation, which initially prohibited formente taught for several years before buying this school after former profit education organizations, now would require that such entities make available to the owner and principal Maria Ester Gajardo Martinez passed away. Test scores public information on their profitability as well are low compared to schools with similar students. as their use of voucher funds.
Data and Methodology
Our study is based on student-level data from Chile’s national standardized test, Sistema de Medición de la Calidad de la Educación (Educational Quality Measurement System—SIMCE), which assesses students in grades 4, 8, and 10 in language, mathematics, history and geography, and natural sciences. In 2002, SIMCE evaluated 274,863 4th graders. Complementing student test scores are parent and teacher questionnaires, which include socioeconomic and environmental information on the students, their families, their peers, and their schools.
socioeconomic characteristics, including parent schooling, self-reported household income, the number of non-school books in the home, and the quality of the peer groups (calculated by averaging family background and home resources for all students in the classroom). We also included some school-level control variables—whether or not the school was located in a rural area, the total number of students per school, and the average monthly tuition a school charges. If these variables fully account for differences in student and peer demographics across the various categories of schools,
www.educationnext.org
S U M M E R 2 0 0 8 / E D U C AT I O N N E X T
65
Vouchers a Boon for Privately Run Schools (Figure 2)
Enrolling just 15 percent when the voucher system began in 1981, by 2002 Chile’s privately run voucher schools educated nearly 40 percent of the country’s schoolchildren.
Percentage of national enrollment, 1981
6.9% Private independent 15.1% voucher 12.7% Network voucher
Percentage of national enrollment, 2002
8.8% Private independent
78% Municipal
25.4% Stand-alone voucher
53.1% Municipal
NOTE: Available data for 1981 do not break down privately run voucher schools by whether they were part of a network.
SOURCE: Education Statistics, Ministerio de Educatión de Chile (MINEDUC), 2002
then this strategy will provide unbiased evidence on the relthey attend that result from the types of schools available to ative effectiveness of municipal, stand-alone, and network them. More specifically, we assume that an individual’s probschools. We cannot account for other factors that could be sigability of choosing a given school type is affected by the school nificant. For example, the average student attending a privately density (that is, the number of schools per square kilometer) run voucher school, whether network or stand-alone, may have of each type in her municipality. All else being equal, students parents who place a higher value on education than those of are more likely to choose schooling alternatives that are more the average student attending a municipal school. Because we densely concentrated in their municipalities. The crucial do not have a measure of parent commitment to education, assumption made by our method is that school choice is we may confuse the effect of havinfluenced by local school supply, ing a committed parent with that but school densities at the commuof attending a private school. Simnity level do not directly influence ilarly, the “brand name” value student achievement. attached to network schools may Though every precaution has enable them to select more-qualbeen taken to make the compariified students, on average, than son exact, it is still possible that our their independent counterparts. results overestimate the benefits As a result, simple comparof privately subsidized schools isons of student outcomes in over municipal ones. For exammunicipal, stand-alone, and netple, it is possible that voucher work schools might give misleadschools are to be found in greater ing estimates of the impact of density in higher income areas, schools on student achievement, All else being equal, students are more likely to choose where more parents are willing to even after adjusting for the mea- schooling alternatives that are more densely concentrated pay additional fees for their chilsured characteristics of the stu- in their municipalities. dren to attend higher-quality dents who attend each type of schools. To the extent that is hapschool. In order to correct for this selection bias, we restrict pening, our results could be biased toward finding greater our analysis to differences across students in the type of school voucher benefits than is actually the case. For that reason, our
66
E D U C AT I O N N E X T / S U M M E R 2 0 0 8
www.educationnext.org
research
CHILEAN SCHOOLS ELACQUA ET AL.
The More Schools, the Better (Figure 3)
Fourth-grade students in network schools outperformed students in stand-alone schools on 2002 math and Spanish language tests.
.60 Spanish .50 .40 Standard deviations .30 .20 .10 0 -.10 -.20 -.14* Municipal Stand-alone Network of 2 schools Network of 3 Network of 4 Network of 5 Network of 6 or more -.07 .00 .00 .20* .14 .24* .37* .39 Math .50* .36* .23*
.34*
.20
Type of school attended
NOTE: * indicates statistical significance at the 95% confidence level
SOURCE: Authors
comparisons between private and municipal schools should be interpreted cautiously. However, that potential source of bias is unlikely to affect comparisons between stand-alone private schools and network ones, the main focus of this analysis.
Results
We report our results in terms of standard deviations of student test scores. The difference in performance between American 4th and 8th graders on the National Assessment of Educational Progress is about one full standard deviation, suggesting that students improve by one quarter of a standard deviation each year. Although comparable measures of the rate of student learning are not available for Chile, researchers studying the Chilean school system typically consider a difference in student achievement of 10 percent of one standard deviation to be a small to moderate effect. Without accounting for any differences in students’ socioeconomic status, the Spanish language and mathematics test scores of students who attend network schools are considerably higher than the scores of those attending standalone schools. After controlling for student and peer attributes and for selection bias, we still find a substantial positive and
statistically significant effect of attending a network school on student achievement. Students at network schools score 19 percent and 25 percent of a standard deviation higher than students at stand-alone schools in Spanish language and math, respectively. We also find that students at municipal schools do significantly worse than students at stand-alone schools on achievement tests (19 percent and 16 percent of a standard deviation in Spanish language and math, respectively), although, as discussed above, we are less confident in
After controlling for student and peer attributes and for selection bias, we still find a substantial positive and statistically significant effect of attending a network school on student achievement.
these results because of the difficulties of accounting for the selection of students into and by private schools. Although these results provide some evidence of the effectiveness of school networks, a more precise analysis is needed to understand the optimal size of a network. We examined whether larger networks are more effective than smaller ones
www.educationnext.org
S U M M E R 2 0 0 8 / E D U C AT I O N N E X T
67
and found that, both with and withPolicy Implications This paper compares the academic out correcting for student and peer achievement of 4th graders in municsocioeconomic characteristics and ipal schools, stand-alone schools, and selection bias, students at schools that network schools. Controlling for indiare part of networks of three or more vidual and peer characteristics as well schools consistently outperform stuas selection bias, we find that students at schools in networks of only dents in network schools outpertwo schools. formed those in stand-alone schools Figure 3 shows the results from The superior performance of network schools is not in both Spanish language and math. our estimations. Students in schools driven by the number of them that are Catholic. The stand-alone schools outperin larger networks generally learned formed municipal schools but not by as large a margin. It also more than students in stand-alone schools. The results for is of interest that students generally performed better in netSpanish language achievement show students in schools in works of large size. Most clearly, those in networks that connetworks with three schools learn 24 percent of a standard tained three or more schools generally outperformed those in deviation more, those in networks of five schools learned 50 pernetworks with only two schools. cent of a standard deviation more, and those in networks of more Possible explanations for the positive school network effect than five schools learned 23 percent of a standard deviation include the substantial benefits of scale for employing educamore. The effects on mathematics achievement are similar. tion professionals and administrators, the bulk purchases of supStudents who attend schools in networks with three schools learn plies and equipment, and the costs of implementing innovations 37 percent of a standard deviation more than students in standin the curriculum. School networks may also benefit from greater access to credit and private investment than that extended to small individual schools in Chile. In addition, it may be that operating within a larger communal organization reduces agency problems; encourages interactions between parents, teachers, administrators, and students; and influences the development of professional school communities. Of course, it is also possible that good schools are invited to join networks, while weaker schools are left on their own. In a competitive schooling environment, lowquality schools may be unable to attract students and additional resources needed to expand operations. alone schools. The percentages for those in networks of five and The results of this paper add evidence to the debate in the more than five schools are 36 and 34, respectively. United States over the desirability of creating networks of Prior research in Chile and in the United States has charter and voucher schools. The findings provide some demonstrated that, all else being equal, Catholic schools outground for optimism about the effects of networking on stuperform public schools and other private schools. Since some dent achievement. Policies that provide incentives for schools of the network schools were affiliated with Catholic churches, to establish a network or to be managed by an organization that fact could be the explanation for the apparent positive that runs a network of schools may have the potential to benefits that come from networking. To determine whether increase educational outcomes. that was the case, we checked whether the school owners were Gregory Elacqua is professor at the Universidad Diego Portales in Catholic. Only 13 percent of the students attended such Santiago, Chile, and former policy advisor to the Minister of Eduschools, however. And after adjusting for Catholic affiliacation of Chile. Dante Contreras is senior researcher at the United tion, the differences between network and stand-alone schools Nations Development Program (UNDP) Chile and associate proremained large and significant. In other words, the superior fessor, Universidad de Chile. Felipe Salazar is researcher at the performance of network schools is not driven by the numUniversidad Diego Portales. ber of them that are Catholic.
Possible explanations for the positive school network effect include the substantial benefits of scale for employing education professionals and administrators, the bulk purchases of supplies and equipment, and the costs of implementing innovations in the curriculum.
68
E D U C AT I O N N E X T / S U M M E R 2 0 0 8
www.educationnext.org
Interested in Education?
Wonder what's happening in our schools?
Education Next gives you:
• • • • Cutting-edge research by famous scholars Provocative features Honest exchanges of opinion Great graphics and engaging text
To subscribe visit
www.educationnext.org or call 1-800-935-2882